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Abstract 

A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the 
pair-wise sum of probabilities of system conditions that have to be distinguished during the 
diagnosing process. The proposed measure describes the uncertainty of the system conditions, 
and at the same time complexity of the diagnosis problem. Treating the assumed combinatorial- 
diagnostic entropy as a primary notion, the information delivered by the symptoms has been 
defined. The relationships have been derived to facilitate explicit, quantitative assessment of the 
information of a single symptom as well as that of a symptoms set. It has been proved that the 
combinatorial-probabilistic information shows the property of additivity. The presented measures 
are focused on diagnosis problem, but they can be easily applied to other disciplines such as 
decision theory and classification. 

Index Terms-- entropy, fault diagnosis, information, multi-valued model, uncertainty 

I. INTRODUCTION 

Constructing an optimal set of diagnostic symptoms/tests and optimal sequence of 
gathering/execution thereof is one of the most important problems in engineering systems 
diagnosis [1] - [7]. Applied optimization method depends on the form of diagnostic model and 
optimization criterion [1],[2],[5],[8] - [12]. 

The diagnostic model describes the relationship between a system condition (the set of faults and 
healthy condition) and diagnostic symptoms [4], [5], [13] - [16]. 

Most models use binary (good - no-good) conditions and binary (normal - abnormal) symptoms 
[3], [8], [17]. Better results can be obtained using the qualitative, approximate, and multi-valued 
models [2], [4], [15], [18], [19]. In [2], it is proved that the length of diagnosis algorithm in the case 
of multi-valued system conditions and multi-valued symptoms is not larger than the binary 
algorithm. 

Qualitative and multi-valued models can be applied to determine a diagnosis algorithm 
[1],[2],[19], approximate inference within expert systems [4],[15],[19]. 

One of the methods of constructing a diagnosis algorithm consists in applying the information- 
based analysis [2], [3], [8], [16], [18], [20] - [24], that is, description of the system condition 
uncertainty and the amount of information delivered by means of individual symptoms and sets 
thereof 

This aim can be reached with the Shannon-introduced quantities: the entropy, and the amount of 
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information [25]. There are some other kinds of entropies [27] that can be considered - Ren5d's 
entropy [26], [28], structural a-entropy [29], and functions z^{t) [30]. Characterization of 

information measures (from the information theory point of view) have been extensively 
discussed in [31], [32]. 

The abovementioned entropies were introduced for solving information-theoretic problems (e.g. 
coding). For other problems, different forms of entropy might be more suitable [26], [28], [33]. 
Therefore, finding the form of entropy best tailored to meet the diagnostic requirements is 
justified. 

This article deals with a multi-valued diagnostic model that exploits the multi-valued system 
conditions and multi-valued symptoms, where the set of values taken by conditions and 
symptoms is finite. 

The set of desirable properties of the proposed diagnostic entropy is determined by taking the 
diagnostics point of view into account. The problem is formulated in a system condition-set 
partition framework [33]. 

The organization of the article is as follows. In section II, basic assumptions concerning multi- 
valued diagnostic model are stated. Section III describes information-theoretic, set-partition 

framework for the diagnosis algorithm designing. It is a starting point for establishing a set of 
postulated properties of diagnostic entropy that is described in Section IV. 
In Section V, the combinatorial-probabilistic diagnostic entropy is introduced and its postulated 
properties are proved. 

The combinatorial-probabilistic diagnostic information of symptoms and sets thereof is defined in 
Section VI. 

II. ASSUMPTIONS 

Further consideration is conducted with the following assumptions formulated and referring to a 
multi -valued model of the system under the diagnosing. 

1. A finite set of the system conditions is determined: 

E = {e,}, i = l,...,n (1) 

Elements e. e E can be treated as random events of the type: "a system under diagnosis is in the 

i-th condition", whereas the whole set E - as a certain event. Conditions might be one-fault or 
multi-fault ones. 

2. The system can remain in one, and only one, of the condition e. e E with the probability -?(£, ) 
, and the following holds: 

\/Pie,)>0, P(E) = 1 (2) 

Probabilities P(e.) estimations can be provided by real -world experiment or by using the well- 
known engineering procedure of the Failure Modes and Effects Analysis [34]. 

3. Determined is a finite set of symptoms: 

D = {d^,r = \,...,t (3) 
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and a finite set of the symptoms' values 

A = {0,K,l-l} (4) 
Multi-valued operator R{.) mapping ExD into A is crisp, fuzzy, or rough one 
[2],[18],[35],[36]. 

If the system is in the condition e. , a value of the symptom is denoted in the following 
manner: 

i?«/e,)=^,,,g,,e^ (5) 

4. For all the symptoms, the following holds: 

V V 3 P[RidJe,) = aJ = l (6) 

It means that diagnosis is noiseless [37] or diagnostic model consists of all real -world instances 
representing class of condition e. e . 

5. The multi -valued model of the system has been presented in the form of a diagnostic matrix 
G: 

G = [g.L, (V) 

where: 

g,,=i?(J,./e,), g^^eA (8) 

The assumed form of the model and the assumptions 1-5 offer good representation of many 
actual systems under the diagnosing [2], [5] - [7], [17], [38], [39]. 

III. AN INFORMATION-THEORETIC, SET-PARTITION FRAMEWORK FOR 

DESIGNING A DIAGNOSIS ALGORITHM 

If the system can remain in any of the conditions e. gE with the probability -P(e.), the Shannon 
entropy H(E) thereof is expressed with the following relationship: 

H(E) = -f^P(e,)\og,Pie,) (9) 

Value A - 2 is usually assumed as the base of a logarithm. The general form (9) is more suitable 
and convenient, if the multi-valued {A > 2) assessment of values of symptoms is a requirement. 

The relationship (9) describes the initial uncertainty of the system condition, that is, prior to the 
selection of any symptom. Assume that any d^e^D of the symptoms has been selected as the 

first one. The set of values thereof takes the form (4). Therefore, subsets Ej(d^)(^E can be 

distinguished. 

V Ej(d^) = {e. ■.R{dje.) = j; ^,=1,L ,n,, j ^A) (10) 

j=0^ ,1-1 

To simplify the notation, indices have been assigned to the subsets Ej(d^) c the indices being 
consistent with suitable values of the symptom d^e^D . 
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On the assumption (6) that there is full likelihood of the value of the symptom gD, the 
following are satisfied: 



a) V Ejid^)nE,id^) = 0, l]Ejid^)^E 

j,l=0 Z-l j=0 

.1 n ^^^^ 

J=0 j=0 ij=l 

What follows fi-om (11) is that the selected symptom d^ induces conditions set partition, which is 
denoted in the following manner: 

{E.(d,)} = {E,(d,),L ,E,_,(d,)} (12) 
If the symptom d^ takes the value j then according to (10) and (11), the system can find 
itself in one of the conditions e:Ej(d^) with the probability: 

P(e, ) 

QieJ = ^ (13) 

Pj 

Provided that the actual condition e* of the system has been identified with accuracy to the 
subset Ej(d^) , the system entropy equals: 

HiE I E.{d^)) = -f^Qie, ) log, Qie, ) (14) 

Because the symptom d^ can take any value j ^ A with the probability Pj , the average entropy 
after having selected the symptom d^ equals: 

H(E/d,) = f^PjH[E/Ej(d^)] (15) 
where: H{E I d^) - the average entropy after having selected the symptom d^ . 

The amount of information on the system condition contained in the symptom d^ is described as 
the difference between the entropies H{E) and H{E I d^.) : 

J(d^) = H(E)-H(E/d^) (16) 
As the conditional entropy H(E / d^) is not larger than the initial entropy H{E), the information 
J(d^) is a non-negative quantity: 

Jid,.)>0 (17) 

After having selected the first symptom, the conditional entropy H(E / d^) is usually larger than 

zero. Therefore, selecting further symptoms proves indispensable. The amount of information 
entered with the symptom selected in the k-th turn is found fi-om the following general 
relationship: 

J(4,)) =H(E/D,_,)-H(E/D,_„d^,^) (18) 

where: 

d^i^^ - S5miptom selected in the k-th turn. 
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- a set k-1 of symptoms selected prior to symptom J^^^ . 

The information-based method consists in subsequent selections of symptoms that contain 
maximum information on the system condition: 

The process is stopped when the remaining entropy is equal to zero or the amount of information 
of all remaining symptoms equals zero. 

One should emphasize that the terms diagnostic entropy and diagnostic information are not the 
same. Diagnostic entropy is a feature characterizing uncertainty of the system condition and/or 
complexity of diagnosis algorithm constructing process. Diagnostic information characterises 
quality of diagnostic symptoms/tests. 

The above-presented information-based method of finding a set of symptoms proves to show at 
least two features of significance: 

• the criterion of selecting the symptoms is defined on the basis of a primary term, that is, the 
entropy, which renders the possibility of gaining coherent measures of the uncertainty of the 
condition and the amount of information entered by the symptom; 

• it uses the property of information additivity - the total amount of information on the system 
condition, gained after the set of ^ symptoms has been selected, equals the sum of conditional 
information of individual symptoms: 

j(z)j=|;j^,,/A-,) (20) 

However, it appears that the application of the criterion function of selecting the symptoms in the 
form of the amount of information does not lead to optimal solutions, even in some specific 
cases, when the set of available symptoms is unlimited. 

On the other hand, the idea of information-based approach to the question of optimizing the set of 
symptoms seems worthy of notice because of the above-mentioned features. A question arises: Is 
it possible to apply a similar approach on the grounds of some function different than the 
Shannon entropy, a function that describes the uncertainty of the system condition; and if so, 
what properties should it then have? 

Additivity is the basic feature of the Shannon entropy, which determines the logarithmic form 
thereof For two statistically independent systems this feature can be written in the following 
form: 

H{E)=H{E,)+H{E,) (21) 
where: H{E) - entropy of a complex system O ; 
H{E.) - entropy of the system O. . 

The two systems are statistically independent if acquisition of information on the condition of 

one of them does not affect the probability distribution of conditions of the other system. 

In diagnostics, the fact of the O, and systems remaining independent can be interpreted as the 

symptoms observed with one of the systems remaining independent of the condition of the other 
system. 

Such being the case, the question of condition identification of a complex system resolves itself 
into two independent questions of condition identification of component systems. Finding a 
solution to the issue formulated in this way does not require the measure of the uncertainty of the 
complex system condition to be defined; therefore, there is no need to satisfy the condition of 
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additivity. 



What results is a conclusion of great significance to further considerations: the function that 
describes the uncertainty of the condition of the system under diagnosis does not need the 
additivity feature in the sense described with (21). This permits to quit the function that shows the 
logarithmic form, that is, the Shannon entropy. In this way, the first of the above-formulated 
questions has been affirmatively answered. 

The question of properties reads as follows: What properties should a function that describes the 
uncertainty of the condition show to make it a basis for the information-like approach for 
optimizing a set of symptoms? And what form should this function take? 

The function searched for will be further on called the combinatorial-probabilistic entropy and 
denoted with H^{E) . 

IV. POSTULATED PROPERTIES OF THE COMBINATORIAL-PROBABILISTIC 

DIAGNOSTIC ENTROPY 

First and foremost, it should be noted that the combinatorial-probabilistic entropy Hg{E) will 

find its application when designing the diagnosis algorithm. Therefore, this measure should take 
into account specific features of that process. In particular, it should facilitate quantitative 
assessment of the partition of the conditions set induced by the selected symptoms. 
A set of postulated properties of H^{E) can be found with its 'conceptual' convergence with the 
Shannon entropy. 

As it follows from (9), probabilities of the system conditions ) are arguments of the function 

of entropy. In some indirect way, this function also depends on the cardinality of the conditions 
set n = card (E) . This dependence becomes more evident if the probabilities of all the conditions 

are P{e.) = l/ n, / = 0,K ,n. Such being the case, the function of entropy takes the following 

form: 

H(E)=log,n (22) 

However, what most significantly affects the value of entropy is the probabilities of the 
conditions. This can be illustrated with the exemplary two systems: the sets of conditions thereof 
as well as probabilities of the conditions are defined in the following way: 

E = {e„e,} P(e,) = P(e,) = 0,5 

P(e[) = P(e^) = 0.05, P{e^) = 0,84, 
P{e,) = Pie,) = 0,03 

Entropies of these systems, at A = 2, are H(E) = 1,0 and H(E') = 0,947 , respectively. It means 
that the entropy of a two-condition system is higher than that of a five-condition system. On the 
other hand, well known is the fact that one symptom is enough to identify the condition of the 
first system, whereas identification of the second system condition needs at least three symptoms. 
Therefore, it seems reasonable to postulate that the demanded combinatorial-probabilistic entropy 
Hg{E) be at the same time the function of the cardinality of the conditions set n = card{E) and 

probabilities of conditions P{e^) : 
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H,(E)=f(n,P) (23) 

where: P = {P(e.)}, i = 1,K , n 

What else should be expected is the monotonic increase in the value of function (23) with the 
growth of «: 

f{n,P)>f{n\P')<^n>n' (24) 
Two other properties result from conditions of setting H^{E) to zero. 

If it is known a priori that the set of system conditions is one-component only ( n = 1 ), the 
condition of the system is then definitely determined and Hg(E) should take value zero: 

^«(^L=i=0 (25) 

Another extreme case takes place when the selected symptoms induce partition of the conditions 
set in the form of one-component subsets {{e,}}, / = 1,K « . It means that all the pairs of 

conditions have been distinguished by the selected symptoms; therefore, the uncertainty of the 
condition equals zero: 

H,(E/{{e,}}) = (26) 

The demanded function is imposed on with the condition of linearity against the assembly of 
variables P in the following form: 

f(n,cPie,XcPie,X...,cPieJ) = cfin,cP) (27) 
where: c is the nonzero constant. 

Because the property (27) has no equivalent among properties of the Shannon entropy, more 
detailed consideration of resulting consequences seems indispensable. Assume that given is some 
partition of the conditions set in the form (12). If an actual condition e* of the system is 
identified with accuracy to the subset Ej , the uncertainty of the condition can be written in the 
following manner: 

H,iE/E.)=fl^nj,Q^) (28) 

where: 

JP(e^ Pi^ Pis) 

[ Pj Pj Pi 

With (27) taken into account, the relationship in (28) takes the following form: 

H,{EIE.) = —f{n.,Pj) (29) 
Pj 

or 

PjH,{EIE.)^f{nj,Pj) (30) 

where: 



Pj={P{e,\P{e,),...,P{eJ 
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The average uncertainty of the system condition, at the partition in (12) assumed, results from the 
definition of the statistical mean value: 

H,{EI{E.}) = f^p.H,{EIE.) (31) 
With the following notation introduced: 

H,(E.) = f(nj,Pj) (32) 
and (30) taken into account , the relationship in (3 1) can be finally written in the following form: 

H,iE/{E.}) = Y,H,(Ej) (33) 

7=1 

The relationship (32) describes the 'contribution' of the j-th subset Ej to the average uncertainty 
of the system condition Hg(E / {Ej}). Consequently, Hg(.) at each stage of the optimization 
process can be found by means of the probabilities a priori P(e.) , with no requirement to find 
conditional probabilities Q(e.) . In other words, imposing condition (27) on the Hg(.) , means 

some reduction in labor demand while carrying out computations to determine the optimal set of 
S5miptoms. 

V. THE COMBINATORIAL-PROBABILISTIC DIAGNOSTIC ENTROPY 

It follows from the above-presented considerations that the combinatorial-probabilistic entropy 
H^{E) of the system under diagnosis should have the following properties: 



H,iE) = f(n,F) (34) 

Hs(E)>Hs(E')<^n>n', n = Card{E), n ' = Card(E') (35) 

^^(^U=0 (36) 

m 

H,(E/{E.}) = Y^H,iEj), Card{{E.}) = m (37) 

7=1 

H,{EI{{e,}}) = Q,i = \,...,n (38) 



The form of the function f(n,P) can be defined with two methods. The first one consists in the 
formal deduction of the function on the basis of the set of postulated properties. This approach is 
followed when there is significant proof that there exists only one function showing the preset 
properties. The second method, usually much simpler, consists in the arbitrary acceptance of a 
certain form of the function and in proving that it shows the postulated properties. 
As there are no limitations put in this article on the number of functions that satisfy conditions 
(34) - (38) the latter of the methods is applied. 

It is worth emphasising that the existence of some 'logical' relation between the combinatorial- 
probabilistic entropy Hg(.) and the process of planning the diagnostic algorithm should be a 
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reasonable prerequisite for the selection of the form of . 
Further consideration is based on the following theorem: 

Theorem 1 
If given are: 

a) a finite set of system conditions 

E = {e^}, i = l,...,n (39) 

b) probabilities of conditions 

VP(e,)>0 (40) 

with 

P(^) = Jp(e,) = l (41) 

i=l 

then function 

H,(E) = Y^f^iP{e,)+P(ej)) (42) 

1=1 j=i+i 

which determines the sum of probabilities of all unordered pairs of system conditions shows the 
postulated properties (34) - (38). 

Proof (draft) 

To prove Theorem 1 lemmas 1 6 are used. 

Lemma 1 

If assumptions (39) - (41) have been satisfied, the following relationship proves true: 

X Z (P(e,)+P(ej)) = XP(e,)(n -I) (43) 

i=i j=i+i i=i 

Proof (see APPENDIX) 

On the grounds of the Lemma 1, the relationship in (42) can be presented in the equivalent form: 

H,iE) = f^Pie,)in-l) (44) 

(=1 

Lemma 2 

Equivalence (35) holds for the function (42). 
Proof (see APPENDIX) 
Lemma 3 

For a one-condition system, function (42) takes value equal to zero. 
Proof (see APPENDIX) 
Lemma 4 
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If given are: 

a) a proper subset EjC^E , such that: 

Ej={e,), ij=l,...,nj (45) 

b) probabilities a priori of the conditions: 

\fP{e.)>(), f^P{e.) = p. (46) 

then the following equahty holds: 

ii X (P(e,^) + P(e,^)) = Pj(nj-l) (47) 

Proof (see APPENDIX) 

On the grounds of the Lemma 4, the following notation can be introduced: 

HBiEj) = f^ X (P(e,) + P(e,))^/..(«.-l) (48) 

ij=\ kj=ij+l 

Lemma 5 

If given is some partition of the conditions set {Ej}, j = l,...,m such that: 

V llP(^) = Pj (49) 

J=l^,m ij=\ 

m 

YuYuP^^i)^^ (50) 

m 

Yn,=n (51) 

7=1 



H,{EI{E.}) = Y^H,{E.) (52) 

7=1 



then the following equality holds: 

Proof (see APPENDIX) 
Lemma 6 

If the partition of the conditions set is given in the form of one-component subsets 
{{e.}}, i = \,...,n , then the uncertainty of the system condition equals zero. 

H,(E/{{e,}}) = (53) 

Proof (see APPENDIX) 

What has been shown by means of the lemmas 1 - 6 is that function (42) shows the postulated 
properties (34) - (38), which completes the process of proving Theorem 1. 

It comes from the above-presented consideration that function (42), which defines the sum of 
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probabilities of all unordered pairs of conditions of the system under the diagnosing, can be 
accepted as the combinatorial-probabilistic entropy. 



VI. THE COMBINATORIAL-PROBABILISTIC DIAGNOSTIC INFORMATION 

The initial uncertainty of the system condition (prior to the selection of any symptom) is equal to: 

H,{E) = n-\ (54) 

If any symptom gD , with its set of values as in (4) has been selected as the first one in the 
sequence, it induces the partition: 

{E.(d^)} = {E,(d,),L ,E,_,(d^)} (55) 
With full likelihood of the value of symptom d^. satisfied then: 

a) V Ej(d^)nE,(d^) = 0, l]Ej(d^) = E 

77=0,.. .,1-1 J=0 

T'l .1 ^^^^ 

b) Xn,=n, X;?,=l, Pj=ZP{e,^ 

In compliance with the Lemma 4, the uncertainty of the system condition, after selection of 
symptom d^ that induces partition (55), equals : 

m 

H,(E/{Ej}) = 2pj(nj-l) 

./=i 

As the partition (55) is explicitly defined by means of the symptom d^ that induces it, the above 
equation can be written as: 

H,iE/d^) = Y,Pjinj-l) (57) 

It is easy to observe that the uncertainty of the system condition after the selection of the 
symptom d^ is not greater than the initial uncertainty, that is: 

H,(E)>H,(E/d^) (58) 
Equality in (58) occurs in the case described with the following condition 

H,(E) = H,(E/d^)^ 3(nj=nApj=l) (59) 

It means that the value of the symptom d^ does not depend on the system condition, and such a 

symptom remains useless as far as the condition identification is concerned. 

On the grounds of equations (58) and (59), the notion of the combinatorial-probabilistic 

information of a symptom can be defined. 

Definition 1 

The combinatorial-probabilistic information of the symptom d^ is equal to the difference in the 

uncertainty of the condition before this symptom has been selected and the uncertainty left after 
the selection. 
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If the symptom is selected as the first one in the sequence, then, according to the Definition 1, 
the following can be written: 

JM)-H,(E)-H,(E/d^) (60) 
where: J^id^) is the combinatorial-probabilistic information of the symptom d^E^D . 

After substituting (54) and (57) into (60) and conditions of (56) taken into account, the 
combinatorial-probabilistic information can be presented in two equivalent forms: 

From (58) and (60) it becomes evident that the information J^id^) can take non-negative values 
only. 

JbW>0 (62) 
The equation (60) and earlier considerations on the form of the combinatorial-probabilistic 
entropy H^(E) give grounds to formulate the following conclusion: 

Corollary 1 

The combinatorial-probabilistic information Jg(d^) is equal to the sum of the probabilities of all 
unordered pairs of conditions distinguishable because of the symptom d^ : 

-/^«)=Z Z ZZ[^H)+^(^..)] (63) 

This confirms the coherence of the introduced measures of the system condition uncertainty and 
the information of symptoms. 

Selection of one symptom usually does not permit condition identification with the required 
accuracy; therefore, there is need for selecting further symptoms. If the d^ symptom, with 
the set of values of the form (4), has been selected as the second one in the sequence, then in each 
of the subsets Ej{d^) of the partition (55) induces the following partition: 

j-o^.aJ^J' « ' } (64) 

where: 

E,M,dJ = {e,, :RidJe.^) = j a R{dje.) = l, 
With full likelihood of the value of the s5miptom d assumed, the following are satisfied: 
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j=0,...,A-l Z,/t=0,...,l-l J'^ JK:^ 
l*k 

1-1 



b) . V U^^,«,<) = £.(JJ (65) 

/=0 

1-1 A-l "ji 

7-0,...,l-l /=() /^,=1 

With the Lemma 4 as the basis and conditions of (56) and (65) taken into account, the 
relationship that detines the uncertainty of the system condition after having selected both the 
symptoms, that is, d^,d^ gD, can be written in the following manner: 

//^ (^ / < , J J = X X (n,, - 1) (66) 

j=0 1=0 

On the other hand, the form of the combinatorial-probabilistic information of the symptom d^ 
under the condition that the symptom d^ has been selected as the first one in the sequence, 
results from the general Definition 1 

J,{dJd^)=H,{Eld:)-H,{Eld^,ds) (67) 

After substituting (57) and (66) into (67), the following is arrived at: 

A,—\ A— 1 A— 1 

Jnidjd^ = Y,pMj -1)- TlLPjAn, -1) = 

./=o /=0 /=0 

i-1 /l-l i-1 

7=0 /=o z=o 

The conditional combinatorial-probabilistic information of the symptom d^ can be written in two 
equivalent forms: 

A-\ /i-1 

■^s (^^. / o = Z Z "y/ - Pji ) 

(68) 

JBidJd;)=Y^ Pji - «,/ ) 

/=0 1=0 

As there is inequality taking place in a way similar to that of (58) 

H,{Eld^)>H,{Eld^,d^) (69) 

therefore, 

J,{dJd^>o 

The above considerations can be generalized to the question of defining the combinatorial- 
probabilistic information of the symptom d^&D under the condition that the earlier selected set 

of A: symptoms D^(^D , then 

A=Hi)'4i)'-'^w} C^^) 

Assuming that after all the symptoms of the set Z)^ have been selected, the partition of the 
conditions set takes the form 

{E.{D,)} = {E,{D,),E,{D,),...,E^^_,{D,)} (71) 
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where: m,^ is the power of the family of subsets, which is the partition of the conditions set E. 
The following relationships also need be satisfied: 

a) m^<n b) m^<A'' (72) 
Equality in (72)a is reached when the symptoms from the set Z)^ induce the (71) partition in the 
form of one-member subsets, whereas equality in the (72)b, when, after selection of each of the 
symptoms, the cardinality of the family of subsets, increases A -times. The uncertainty of the 
system condition after having selected all the symptoms from the set Z)^ equals: 

ni/- -1 

H,iE/D^) = Y^p.inj-l) (73) 

If the symptom ^ is selected as the next one successively, then it induces partition in the 
form of (74) in each of subsets of the partition (71) : 

. V {E.,(D„dX..,Ej,_,(D„dJ} (74) 

j=0,...mi,-l ■' ■' 

where: 

Eji(DtA) = {% ^E:R(dJe.^^ =/, /.^ =l,...,n^.;} 

With full likelihood of the value of the symptom d^ assumed, the conditions similar to those of 
(65) are satisfied, and: 

J-'^ 1=0 1=0 

The uncertainty of the system condition after having selected all the symptoms from the set Z)^ 
and the symptom d^ ^ Z\ is equal to: 

H,{EID„d^) = Y^Y^p.,inj,-\) (76) 

j=0 1=0 

Using the general Definition 1, the combinatorial-probabilistic information of the symptom d^ , 
under the condition that earlier the set of symptoms Z)^ has been selected, can be written in the 
following form: 

J,{dJ D,) = H,{EI D,)-H,{EI D„d^) (77) 
After substituting (73) and (76) into (77) and taking (75) into account, the following is arrived at: 

(78) 

y=o /=o 

It can be easily noticed that relationships (78) are the generalization of (68); they become 
identical when k = l, and = /I . 

Another issue of significance is to define the information of the set of k symptoms Z)^ c £) . In 
order to do this, the earlier infroduced Definition 1 should be generalized to the following form: 
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Definition 2 

The information of the set of symptoms J^{Di^) is equal to the difference between the initial 
uncertainty of the condition H^(E) and the uncertainty left after having selected all the 
symptoms from the set Z)^ - Hg{E / Dj^) : 

J,{D,) = H,{E)-H,{EID,) (79) 

After substituting (54) and (73) into (79), the following is arrived at: 

JbW= Y,nji\-p.) (80) 

7=0 

What comes out from the comparison between (61) and (80) is that both the relationships take 
identical form in the case Z)^ is a one-member set. 

Assuming, that after having selected all the symptoms from the set Z)^ , the condition uncertainty 
is non-zero and the symptom ^ has been selected as the next one in the sequence. The joint 
partition of the set of conditions induced with the symptoms from the set D,^ and ^ Z\ has 
taken the form (74), whereas the condition uncertainty is described with the relationship (76). 
Using Definition 2, the total information of the set of symptoms and the symptom d^ ^ Z\ 
can be presented in the following form: 

J,(D„dJ^H,(E)-H,(E/D„d^) (81) 

After substituting (54) and (76) into (81) and transformation with the (75) taken into account, the 
following is arrived at: 

A(A.^J= im^jii^-Pji) (82) 

j=0 1=0 

What results from the above-considered issues will be used to prove the Lemma 7, and Theorem 
2. 

Lemma 7 

The total information of the set of symptoms Dj^c^D and the symptom d^ ^ JD^ is equal to the 
sum of the information of the set Z)^ and conditional information of the symptom d^ 

J,{D„d^)= J,{D,)+J,{dJD,) (83) 

Proof (see APPENDDC) 
Theorem 2 

The information of the set of symptoms Z\={i/^}, k = \,...,K, D^^ c^D equals the sum of 
conditional information of individual symptoms. 

J,{D^)= X/,(c/(,3/A-i) (84) 

Proof (draft): 

The proof is carried out with the induction method. 
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For ^ = 1 

L(84)=J^(A)=J^^i)) 

that is 

L(84)= R(84) 

Assuming that (84) proves trae for any 1 < M < card(D) 



UDm)= (85) 



k=l 



After having added J^id^M+x) I A/) ^^^h sides of (85), the following is arrived at: 



M 



k=\ (oO) 

Using the Lemma 7, equation (86) can be brought to the following form (after the transformation 
of the right side thereof): 



M+l 



k=l 

This, according to the principle of induction, makes the proof of the theorem complete. 
The proof has shown that the combinatorial-probabilistic diagnostic information has the property 
of additivity. If the amount of information delivered by set of symptoms equals to the entropy 
value then system condition is identified with necessary accuracy. 

APPENDIX 

Proof of Lemma 1 (draft): 

After simple transformations on the left side of (43), and changes in summation indices, the 
following is arrived at: 

Z(43) = XP(e,)(n-l) (87) 

i=l 

The comparison between (43) and (87) shows that: 

Z(43) = 7?(43) q.e.d. 

Proof of Lemma 2 (draft): 

On the basis of (44), the following can be written, respectively: 



a) H,{E) = Y^P{e,){n-\) 

(88) 

1=1 

After subtracting the sides of equations (88)a and (88)b, and after the transformation with 
assumption (41) taken into account, the following is arrived at: 

H,(E)-H,(E')^n-n' (89) 
The following equivalence directly results from the relationship (89): 
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(E)>Hg(E')^n>n' q.e.d. 

Proof of Lemma 3 (draft): 

After substituting n = 1 into (44) the following is arrived at: 

H,(E)l^,^0 q.e.d. 

Proof of Lemma 4 (draft): 

On the grounds of the assumptions (45) and (46), the left side of (47) after suitable 
transformations, is arrived at: 

Li47) = pj(nj-l) (90) 

The comparison between (47) and (90) shows that: 

Z(47)=7?(47) q.e.d. 

Proof of Lemma 5 (draft): 

The average uncertainty of the condition, for some assumed partition, can be found from the 
definition of the expected value: 

m m 

H,(E I {E.]) = Y^p.H.iEle ^E.) = J^p.^n. -1) (91) 
7=1 y=i 
On the other hand, on the grounds of the Lemma 4, after the summation as related to all j, the 
following is arrived at: 

m m 

t.H,{E.) = Y^p.{nj-\) (92) 
The comparison between (91) and (92) shows that: 

m 

H,{EI{E.}) = J^H,{E.) q.e.d. 

7=1 

Proof of Lemma 6 (draft): 

With the Lemma 5 employed, the uncertainty of the condition can be written in this case in the 
following form: 

H,{EI{{e,]]) = Y^p.{n.-\) (93) 

7=1 

After substituting rij =1, j = \,...,n into (93), the following is arrived at: 

H,{EI{{e^)}) = Q q.e.d. 

Proof of Lemma 7 (draft): 

With (80) and (78) as the basis, the following can be written, respectively: 

(94) 

mt-l /l-l 

b) UdJD,)^ ZZn.XPj-Pj,) 

j=o 1=0 



17 



Having summed up (94)a and (94)b by sides, 



J,{D,) + J,{dJD,) 



Y,nj{l-Pj) + 



m^-l A-l X-\ 



7=0 ;=o 



is obtained. Finally, 



J,{D,) + J,{dJD,) 



(95) 



7=0 1=0 




q.e.d. 
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